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ABSTRACT 



In 1998, the Chicago Public Schools piloted the Chicago 
Academic Standards Exam (CASE) to assess how well students had learned the 
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Frameworks. This report examines the implementation of district curricular 
standards and assessments and the effects on teaching in four high schools. 
Researchers collected district-level curricular standards, materials 
concerning the CASE given to schools and teachers, and the district's 
Programs of Study. They interviewed central office administrators involved in 
developing and implementing the standards and assessments and teachers and 
administrators at the four schools regarding their responses to the CAS and 
CASE. They observed teachers at each school teach the same lesson and 
gathered work from high- and low-achieving students in those classes. Results 
indicated that district-wide standards in English represented multiple 
curricular and instructional goals; stated goals were displaced as standards 
were translated into the Programs of Study and CASE; teachers' curricular 
decisions and instructional choices were very similar across schools; 
teachers responded most to regulatory aspects of the standards and 
assessments (with the CASE exam structuring the curriculum) ; and teachers 
received only limited professional development regarding the standards and 
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The Challenge of Improving Instruction in Urban High Schools: 

Case Studies of the Implementation of the Chicago Academic Standards 

Kenneth K. Wong 
Dorothea Anagnostopoulos 
Stacey Rutledge 
Claudia Edwards 

University of Chicago 

I. INTRODUCTION AND STUDY OBJECTIVES 

Efforts to improve schools through curricular standards and aligned assessments 
have gained increasing prominence over the past two decades. Almost all states have 
curricular standards as well as assessments intended to measure how well students have 
met these standards. In addition, school districts, particularly urban districts, have begun 
to institute their own curricular standards and assessments. In June 1998, the Chicago 
Public Schools (CPS) introduced content-based standards and frameworks at the ninth 
and tenth grade levels in the core academic subjects: math, Enghsh, science and social 
studies. That same year, the district piloted the Chicago Academic Standards Exam 
(CASE) to assess how well students had learned the content and processes delineated in 
the Chicago Academic Standards (CAS) and Frameworks (CAP). 

Standards have been hailed by proponents as levers to improve teaching and 
learning at the district and state-wide levels. The difficulty of this task, particularly for 
urban districts, must be considered if we are to better understand the potential that this 
type of reform holds. To be sure, Chicago has made rapid gains in student and school 
performance in the last four years. Nonetheless, like many other urban school districts, 
Chicago continues to face high rates of student and school failure. At the end of the 
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1998-99 school year, in Chicago high schools, on the average, approximately 50% of 
ninth and tenth graders failed one or more core subjects, while 32.2% of students scored 
at national norms on standardized reading tests. ' District and school efforts to implement 
curricular standards and assessments must be understood within the context of enormous 
academic challenge. 

This report examines the implementation of district curricular standards and 
assessments and the effects of this implementation on teaching in four case study high 
schools in Chicago. In Section III, after explaining our methodology, we conduct an in- 
depth examination of the districts goals and intentions for its standards and frameworks. 
In Section IV, we turn to the question of how teachers respond to curricular standards, 
frameworks and assessments and how they shape instructional and classroom practices. 

In Section V, we delve into how teachers across schools taught the same piece of 
hterature; To Kill a Mockingbird.^ In this analysis, we examine multiple dimensions of 
classroom practice and interaction, ranging from the distribution of classroom activities 
to analyses of teacher-student dynamics. Finally, our conclusion offers policy 
recommendations, including strategies to raise the bar of instructional practices. 

Chicago B standards and assessments are obviously implemented within the larger 
context of the districts accountability agenda. In order to understand the nature of this 
agenda, we develop a differentiated understanding of the Chicago accountability agenda 
to the districts curriculum and assessments pohcy. Table 1 suggests that Chicago B 
accountability agenda entails a mixture of regulation, support, and professional 
discretion. The success of the agenda in improving teaching and learning within the 
districtB schools rests, in part, upon the balance the district strikes among these policy 
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mechanisms. ChicagoB efforts to implement curricular standards and assessments must 
be understood within the context of this larger agenda. 

Within this accountability agenda, the standards and assessments serve as the 
districts main tool to guide curriculum and instruction. A close analysis of the districts 
Programs of Study, the document that turns standards into curricular and instructional 
guidelines, and the CASE, reveals both the intent of the district and how the standards are 
being translated into practice. Through interviews with teachers we assess their 
implementation. We examine teachers [I]perceptions of standards and assessments. 
Through classroom observations and analyses, we look at the choices made by teachers 
and the opportunities provided to students. In light of the district goal to improve 
literacy, develop students Lbritical thinking skills, and set clear curricular expectations, 
our study provides field-based findings into how teachers negotiate these multiple 
demands. 

Summary of Findings 

Clearly the intent of the Chicago Academic Standards (CAS) and the CASE exam 
by the district and its implementation by teachers is complex. The findings from our 
multi-level analysis into the implementation of the Chicago Academic Standards and 
Frameworks and its related assessment, the CASE, follow: 

• Districtwide standards in English represent multiple curricular and instructional 
goals. 

District goals stated in the CAS dociunent serve not only to regulate curricular 
and instructional decisions, but also to support teachers by clarifying key instructional 
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goals and objectives. A close analysis of ChicagoB standards reveals that their scope is 
impressive and represents a cross-section of different approaches to teaching English. 
Standards include a multi-cultural and historical approach to English instruction as well 
as constructivist, and conventional, text-centered approaches. 

• Stated goals are displaced as standards are translated into the Programs of Study 
and CASE. 

A funneling process occurs as the standards and fiameworks are translated into 
documents intended to guide teachers Qnstruction and assessment. While a broad array 
of goals is reflected in the CAS, these are not represented in the Programs of Study and 
the CASE. Those goals that remain tend to be ones that are easily measured and 
assessed. 

• TeachersCcurricular decisions and instructional choices are remarkably similar 
across the four schools in our study. 

Teachers [reported objectives for the instructional unit. To Kill a Mockingbird, 
were remarkably similar across schools. Teachers Qnain goals were to have students 
rmderstand the plot and main themes of the novel. In terms of classroom activities, 
teachers devoted most of their time to literal comprehension of the novel. This is 
reflected in their choice of activities, such as having students read silently or orally and 
reviewing the chosen selection, and in an analysis of the types of questions posed to their 
classes. No time was spent on discussion. 

• Teachers respond most to the regulatory aspect of the standards and assessments. ^ 
The CASE exam is serving to structure the curriculum. 
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The CASE is the dominant mechanism guiding teachers Ciurricular and 
instructional decisions. Teachers are allocating time to the curriculum tested in the 
CASE. They are prioritizing coverage of material over critical thinking skills. Their 
instructional practices reflect a focus on literal comprehension over more complex 
thinking skills. They are practicing writing as modeled by the exam. 

• The district and the schools have provided teachers with only limited support in the 
form of professional development related to the standards and assessments. 

As a consequence, teachers [Ideas on how to respond are developed either 
individually or with fellow department members. While this contributes to curricular 
alignment, it does not lead to teachers experimenting with different instructional 
approaches. In our analysis of instructional activities, we observed very little cooperative 
learning or student-centered activities. Even games, aimed at greater student 
participation, were veiled recitations with teachers asking students literal questions. 

Policy Implications 

• The district needs to be clearer on its goals for the standards and assessments. 

Diverse student academic needs may require differentiated curriculum and 
instruction. 

Is the main goal of the district to align curriculum? Develop higher order thinking 
skills? Guide teachers [Instructional practices? Improve literacy? As the standards are 
transformed into frameworks of practice, many of these goals are lost. As currently 
structured, the standards and assessments are leading to a common denominator of 
instruction. Yet different students have different needs. Our case study schools range 
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from a college prep magnet school to a reconstituted school. Yet, we observed common 
forms of instruction across schools. Students already scoring at grade level should be 
focusing on different skills than those struggling with basic literacy. Yet currently we do 
not see any differentiation between the instruction given to distinct populations. The 
district needs to provide clearer support to teachers based on the unique demands of their 
student population, not only across schools but also within schools. 

• With teachers modifying their instruction to meet curricular demands, both district 
and schools have an opportunity to influence not only teachersOcurricular decisions 
but also their instructional practices. 

Findings from this study indicate that teachers focus primarily on literal 
comprehension and use a limited number of activities in their classrooms. This suggests 
a basic proficiency on the part of the teachers and the district. If the district wants to 
raise the bar of instructional activity, it needs to provide sustained professional 
development, either through district programs or in the schools, aligned with district 
goals modeling an array of approaches. If the goal of the CAS and CASE is indeed to set 
basic proficiency levels, then it is consistent with the current practice. If, however, the 
district wants to raise the bar, it needs to re-evaluate its current approach to presenting 
and preparing teachers for the CASE. 



n. RESEARCH DESIGN 

Chicago B standards and assessments are implemented within a larger 
accountability agenda. Since district policy occurs within a complex organization, we 
took a multi-level approach to examine how the standards and assessments were 
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implemented and how this implementation affected teaching and learning. At the district 
level, we collected curricular standards, copies of materials concerning the CASE that the 
district provided to schools and teachers, and the districts Programs of Study. We also 
interviewed central office administrators involved in the development and 
implementation of the standards and assessments. 

In Older to assess teachers Qcsponses to the CAS and CASE, we interviewed math 
and English teachers in four Chicago public high schools. The high schools have received 
varying levels of district intervention over the course of the last three years. School A 
has never been on probation. School B was taken off probation after one year. School C 
remains on probation for the fourth year, and School D has been reconstituted; teachers 
had to reapply for their jobs at the end of the 1997-98 school year. Table 2 provides 
demographic and test score information for each school. The schools differ in terms of 
the racial and ethnic backgrounds of their students. Their student populations, however, 
are all overwhelmingly low-income. School D has the lowest percentage of students 
scoring at national norms on the TAP reading exam of the four schools, while School A 
has the highest. 

At each school, we interviewed the principal, administrators in charge of 
instruction, the academy resource teacher, and math and English teachers. Interviews 
focused on how individuals understood and responded to the districts standards and 
assessments, and how district policies affected the distribution and use of instructional 
resources. Because the district assessments affect only ninth and tenth grade teachers, 
this study focuses on the interviews we conducted with these teachers.^ 
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In order to assess the influence of the district standards and assessments on 
teaching practices, we observed two tenth grade teachers in each school teach To Kill a 
Mockingbird.^ The district mandates that teachers teach this novel and the tenth grade 
CASE tests students Lknowledge of it. We observed one section of English II taught by 
each teacher and took verbatim accounts of classroom instruction over the course of the 
instructional unit. Audio tapes of classrooms were recorded and transcribed. Each class 
was observed at least eight times over the course of the unit. Because of time constraints, 
we were not able to observe the beginning of the units in School B and School C. We 
collected observations over the course of the second half of the school year. Different 
teachers taught the imit at different times of the semester. 

In addition, we asked teachers to identify two high-achieving students, two 
average-achieving students, and two low-achieving students relative to the class 
observed. We collected all of the work these students completed during the course of the 
unit. Finally, we interviewed the teachers observed to understand in more detail how 
they used district support and documents to plan their instruction, their goals and 
objectives for the unit, and their assessment of the unitB effectiveness with students. 

While we believe that our study provides valuable insight into how Chicago Q 
attempt to implement district-wide curricular standards and assessments affects teachers □ 
instructional practices, there are some limitations to our findings. First, we examined 
only four high schools. Although the schools have encountered a wide range of district 
interventions, the generalizability of our findings is obviously limited. Second, this study 
focuses on only one instructional unit. It does not allow us to see how teachers have 
responded to the districts curricular policy throughout that entire academic year. Taken 
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as a whole, however, our multiple methods are complementary and contribute to a fairly 
comprehensive understanding of the implementation and effects of the districts standards 
and assessments policy as it currently exists. 

m. DISTRICT-WIDE ACCOUNTABILITY STANDARDS AND ASSESSMENTS 
Policy mechanisms in the District Educational Accountability Agenda 

The CAS and CASE represent just one component of the districts educational 
accountability agenda. Other elements of this agenda include an academic promotion 
policy for students that mandates grade retention for poorly performing students, and a 
probation policy that places schools with less than 15% of their students scoring at 
national norms on standardized tests of reading and math under district intervention with 
the threat of reconstitution and possible closure. It also includes policies that specifically 
affect high schools. These high school specific policies center around changes in the 
curriculum and in how students progress through high school. The district has increased 
graduation requirements for high schools students, reduced the number of elective 
courses students can take, and increased math, science and foreign language 
requirements. The district has also designated the first two years of high school as the 
Junior Academy, and the last two as the Senior Academy. Students in the Junior 
Academy (ninth and tenth graders) must earn credits in all of the core academic subjects 
and achieve grade-level equivalent scores on the Test of Achievement and Proficiency 
(TAP) in order to progress to the Senior Academy. According to the districts 1997 High 
School Redesign Plan, students in the Senior Academy enroll in more specialized courses 
and programs, including Career Academies and junior college courses. 
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In previous reports, we examined how these district policies fared in the public 
schools.^ We found that while sanctioning policies, exemplified by the academic 
promotion and probation policies, have drawn the most attention, the districts agenda 
also includes efforts to support school staff and to strengthen teachers Eland principals □ 
professional discretion. Table 1 categorizes the various strategies involved in the key 
components of the districts agenda. Several components of the agenda focus on 
regulating school, teacher, and student behavior and performance. Regulations take the 
form of sanctions, such as grade retention for low-performing students and the threat of 
restaffing associated with probation and reconstitution. Support policies represent district 
efforts to assist schools, teachers, and students to improve performance. One example of 
a support policy is the use of external partners, or consultants, to assist in improving 
instruction in schools on probation and reconstitution. The districts agenda also includes 
components that reinforce school-level discretion. For example, principals and teachers 
have the fineedom to design the Junior and Senior Academies as they see fit. 

To be sure, teachers and school-level administrators have responded differently to 
the regulatory, supportive and discretionary aspects of the districts accountability 
agenda. In our comparison of four high schools with varying degrees of academic 
success, we found that teachers and school-level administrators generally supported 
district efforts that allowed for their professional discretion, such as the academy 
initiative.’ Teachers and principals spoke positively about the Junior Academy structure. 
They also responded to sanctioning policies, particularly, probation. This response, 
however, varied depending on the degree to which the school was subject to, or 
threatened by, probation. Thus while schools restructured teachers dime to include staff 
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development focused on teaching reading strategies and test-taking skills, it was 
primarily those teachers in schools under probation and reconstitution who allocated 
significant amounts of classroom time to these activities. Based on our analyses of 
approximately 200 hours of classroom observations, we found that English teachers in 
the reconstituted school spent 60% of the observed time on test skills development 
activities and 3% of the time on test preparation. In contrast, teachers in the case study 
school that was not subject to any district intervention spent 0% of their English 
instructional time on test practice activities. * 

Teacher response to district support efforts tended to be mixed. Key supportive 
components in the district® agenda included the creation of external partners for low- 
performing schools, district curriculum for student advisories, and funds for teachers □ 
common planning time. In schools under probation and reconstitution, conflict arose 
between teachers and external partners. In effect, what was seen as QupportOby the 
district, was perceived more as a SanctionCby the teachers. Further, the majority of the 
teachers interviewed across the schools rejected district efforts to institute advisories and 
their attendant curriculum. The district® goal for advisories was to provide social and 
personal support to high school students. Teacher resistance to this program resulted in 
the district revising the advisory curriculum to focus more on academic skills and less on 
social and personal issues. 

The Role of Standards and Assessments in the District® Accountability Agenda 
Like its other core policies, the district® use of the curricular standards and 
assessments entails a combination of regulation, support, and professional discretion. As 
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a regulatory measure, district documents indicate that it intends to use CASE and TAP 
results to identify which schools to place on probation in the future. In addition, the 
CASE will be used to determine whether or not students can progress from the Junior to 
the Senior Academy. 

At the same time, the district views the standards as a tool to support teachers by 
clarifying key instructional goals and objectives. The district also seeks to help teachers 
prepare students for the CASE through the creation of the Programs of Study and the 
Structured Curriculum. Both documents represent efforts by the district to help teachers 
incorporate the academic standards into their curriculum and instruction. The Programs 
of Study delineate those curricular standards that are tested on the CASE. In effect, they 
translate the standards into instructional units with set objectives and lengths. The 
Programs of Study for each core subject, then, play a key role in the implementation of 
the districts standards and assessments. The Programs of Study do not specify 
instructional activities or strategies which teachers must apply; they only provide 
suggestions. This, then, allows for professional discretion. In addition to the Programs 
of Study, the Structured Curriculum for each core subject includes imit plans that 
incorporate the districts standards. The district provided all teachers with a Structured 
Curriculum for their courses in the 1999-2000 school year, though it did not mandate that 
teachers use the Structured Curriculum. In other words, instruction is left to teachersD 
discretion. 

In sum, the districts policies concerning the standards and assessments seek both 
to regulate teacher and student performance and to support teachers Cbfforts to Seach to 
the standards. □ They also provide teachers with discretion to select and develop 
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instructional activities within the district framework. Thus the relationship between 
regulation, support and professional discretion is particularly relevant to the 
implementation of curricular standards and assessments. 

Instructional Role of the CAS, CASE and the Programs of Study 

As noted above, the districts implementation of the standards and assessments 
involves regulatory, support and discretionary components. To what extent, however, 
does the districts implementation of the standards and assessments strike an effective 
balance between regulation and support? Do the standards serve only to hold teachers 
and students accountable for specific content, and thus represent a minimum proficiency 
level, or do they provide support to increase teachers instructional capacity? Are there 
signs to suggest that the CAS/CASE provide opportunities for long-term instructional 
improvement? 

Interviews with both teachers and district administrators indicate that thus far the 
district has used the standards and frameworks primarily as an accountability tool. There 
are, however, some supports provided by the district in terms of preparing teachers for 
the CASE. Table 3 summarizes these main forms of district support. The large majority 
of teachers reported receiving the Programs of Study. Most teachers also reported that 
several weeks before the administration of the CASE they received sample questions and 
lists of literary terms. However, teachers did not have the CASE questions at the time 
they developed their curriculum because these questions were only distributed within two 
weeks of the CASE test. Additionally, Table 3 shows that very few teachers reported 
attending any staff development concerning how to teach the standards. Teachers who 
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did attend district meetings on the CASE reported that their function was primarily 
administrativeD providing teachers with information concerning the number and types of 
questions on the exam and how and when to adm ini ster it. Indeed, according to teacher 
interviews, the district did not fully provide teachers with a rationale for the structure and 
design of the CAS/CASE-based curriculum, nor did it systematically provide teachers 
with strategies to incorporate the standards into their lessons or instructional approaches 
associated with the standards. 

These findings suggest that a long-term, sustained focus on instructional 
improvement remains somewhat unclear this early in the implementation process. It is 
important to note, however, that part of the reason for the limited focus on instructional 
improvement is likely due to the Qiewnessnof the policy. As anticipated in the 
implementation literature, organizational adaptation to new goals takes time and 
resources.^ A central office administrator involved in the creation of the standards and 
frameworks noted that there was negligible funding available to district officials to 
provide schools with staff development about the standards and frameworks. In the first 
year of implementation, schools had to request assistance or training from the district. 
This central office administrator noted that few schools would do this given the intense 
scrutiny of low-performing schools put into place by the districts probation policy. Few 
schools would want to draw attention to their need for instructional guidance given this 
pressure. Policies such as these, constrained by a lack of resources, have had a marked 
effect on the balance between regulation and support. At the end of the second year of 
the CASE pilot, one district official noted that the CASE had become the [Jail that 
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wagged the dog;[I]the lack of adequate support for teachers to change their instruction 
resulted in instruction being driven by the assessments rather than the standards. 

Linkage Between the CAS and the Programs of Study 

In light of the limited scope of the districts staff development efforts, the 
Programs of Study and Structured Curriculum represent the districts primary 
mechanisms with which to support teachers in their implementation of the district 
standards. For the purposes of this report, we will focus on the Programs of StudyD the 
first of these tools to be disseminated among teachers. Because the Programs of Study 
serve as the primary means of support the district provides teachers to Qeach to the 
standards,Dit is important to examine how the Programs of Study distills these 
instructional goals. This is particularly relevant as the Programs of Stucfy are intended as 
a tool allowing teachers to create an instructional bridge between the standards and 
frameworks and the CASE exam. As one central office administrator said of the 
Programs of Study, CttB not all of what a teacher should teach. But for the purposes of 
assessments □ this is what we have judged to be the important things your student should 
knowD 

In this study we examine how the district utilizes curricular standards and 
assessments to enforce accountability and to drive instructional improvement. Curricular 
standards and assessments typically serve two purposes. They hold schools and students 
accoxmtable for teaching and learning particular skills and they provide teachers with 
curricular and instructional blueprints. Though most curricular standards, like those 
developed by professional associations such as the National Council of Math Teachers 
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(NCTM) and the National Council of Teachers of English (NCTE), delineate the key 
skills students should master in each subject while leaving instructional matters to 
teachers, they do imply instructional theories or approaches, and serve to support 
particular types of teaching practices. 

What are the goals of the English standards and frameworks for ninth and tenth 
grades? Their scope is impressive, as they touch on an amalgam of various approaches to 
English instruction. Goals that require students to Dtonstruct and extend meaning from 
the textDsuggest a constructivist approach, while objectives that state that students should 
know how to Synthesize and evaluate ideas from various cultures, Dand Qllustrate how 
form, content, purpose and major themes of literary works reflect cultures, literary 
periods and ideas that shaped them. Dimply a multi-cultural and/or historical approach to 
literature. In addition, conventional, text-centered notions of English instruction also 
appear in the standards and frameworks. Students must learn to Evaluate relationships 
between plot and subplot, cormecting themes, character traits, motives, tone, point of 
view and setting in fictional selections of different genres and eras, Dand Do draw 
conclusions concerning the use and impact of plot structure. □ These goals focus on 
students being able to identify the basic elements of different types of literature and non- 
fiction. The three objectives need not be mutually exclusive. Students need to consider 
how authors use structural elements in order to draw generalizations and make 
interpretations. Of central concern, however, is the priority the district places upon these 
various approaches. 

A close analysis of the Program of Study for English that compares the Program 
to the districts standards and frameworks raises questions about the purposes of the 
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standards. The English Program of Study, as currently developed, focuses primarily 
upon a text-centered, conventional approach to English instruction that stresses the 
identification of the elements of different types of literature. The curriculum outlined in 
the Program of Study can be classified as what Applebee, Burroughs and Stevens call a 
DfcollectionsIIltype of curriculum structure, Qvhere the parts that are included are selected 
for study as part of a set, such as Great Books, or Modes of Discourse. d° The Program 
of Study divide the curriculum into units on different literary genres - non-fiction, drama, 
novels and short stories, and poetry. Works are selected because they represent the 
various genres and their elements are studied to identify what constitutes each type of 
genre. This type of curriculum structure provides only minimal coherence and lacks the 
conceptual organization required by the more complex forms of curriculum structure that 
Applebee et al identify as episodic and integrated. 

In short, while the districts standards and frameworks use language that indicates 
the districts intent for teachers to employ constructivist approaches, the Program of 
Study indicates that the district holds teachers accoimtable only for teaching the basic 
literary elements of different types of writing. While the objectives in the Program of 
Study state that students should be able to identify themes, there is no mention of students 
engaging in constructing interpretations of literary works. Constructing interpretations 
entails drawing generalizations and developing arguments to support these 
generalizations. Identifying themes does not imply this type of analysis and elaboration. 

It comes as no surprise, then, that despite the high-level objectives of the Chicago 
Academic Standards (CAS), the classroom teaching we observed, as we shall discuss 
shortly, by and large did not reflect the more challenging skills envisioned in the CAS. 
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This is in part a consequence of a furmeling process through which the expansive goals of 
the CAS are narrowed for the purposes of teacher assistance (in the form of the Program 
of Study) and testing (in the form of the CASE). Because district support is directed at 
facilitating teacher preparation for the CASE exam, it is necessarily the more limited 
goals of the CASE exam that dominate teachers [Iburricular choices. It is clear that the 
exigencies of assessment and accountability have resulted in setting aside those CAS 
goals that are not readily measured and assessed. 

This fimneling process encompasses several stages. As mentioned above, the 
broadest range of goals exists at the abstract level of the standards themselves. At this 
point the funnel is widest because no single curricular model is employed; the CAS 
incorporates numerous approaches to teaching. This variety of approaches is in part 
programmatic and in part attributable to the wide range of participants involved in 
developing the CAS, including teachers, local university partners, the Washington-based 
think-tank NCEES, as well as the Chicago Teachers OUnion. Additionally, various 
models were used as guidesD professional standards like the NCTE and the NCTM as 
well as the Illinois State standards and standards developed by other states. Given this 
wide range of input, the CAS proposes a variety of sound □ and at times innovative □ 
approaches to teaching. At the same time, it is not easy to encompass such a variety of 
directives within a single practical teaching guide and assessment tool. 

It is in this move towards practicality that the fimneling process begins. The 
Programs of Study seek to extract key goals from the CAS and offer suggestions that 
would render such goals tangible to teachers. The aim of the Program of Study, however, 
is not merely to serve as a professional support tool for Chicago Public School teachers. 
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It also serves as the blueprint from which the CASE exam is formulated and therefore 
represents the primary CASE-preparation resource for teachers. Thus, although the 
CASE exam is meant to assess the skills outlined in the CAS, in effect it is measuring 
those skills that were funneled from the CAS into the Program of Study. The Programs 
of Study, then, become the de facto determinant of what skills teachers should be 
teaching. 

At this early phase of implementation, our analysis of the English Program of 
Study suggests that the standards tend to set a basic proficiency level. Because the 
Program of Study provides the teaching guidelines for the CASE exam, it is questionable 
whether teachers are being asked to alter their instructional methods in any substantial 
way. Since, according to district administrators, the English CASE exam generated the 
highest rate of passing scores and the fewest complaints, it seems the success of the exam 
is in part the result of its relatively traditional approach to English instruction. Indeed, as 
one central office administrator said of the English CASE, Ut3 not much of a change 
from what teachers usually do anyway. □ It remains to be seen if the Program of Study 
will be revised to reflect higher expectations and more complex approaches to English 
instruction. 

Analysis of the English CASE 

The English CASE exam assesses students in two categories of achievementO 
basic knowledge, and understanding and reasoning skills. While basic knowledge skills 
suggest recognition of literary terms and elements, understanding and reasoning skills 
suggest a range of text-based skills, from basic literal comprehension to understanding 
and interpreting implied meanings. Of 30 multiple choice questions on the English I and 
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II CASE Sample Exams, 14 (47%) of these are intended as basic knowledge questions, 
and 16 (53%) as understanding/reasoning questions. In addition to the multiple-choice 
section, there are 4 constructed response questions □ 3 short answers and one essay-style 
response. Constructed response questions fall into the imderstanding/reasoning category. 
Within this structure, in order for a student to attain the 50% mastery necessary to pass 
the English CASE, some portion of the understanding/reasoning questions must be 
answered correctly. 

The district piloted the CASE in ninth grade algebra, English, social studies and 
science in June 1998. Central office officials reported that 75.8% of ninth graders passed 
the English CASE, 42.7% passed the history exams, 35.5% passed biology, and 35.5% 
passed the algebra. ' ' According to interviews with district administrators, English 
teachers have been the least resistant to the districts English standards and CASE. If the 
districts standards and assessments were to have an effect on teachers' instructional 
practices, we would most expect to see this effect in English classrooms. 

We analyzed the multiple choice questions on sample exams for English I and 
English II in order to make a more precise determination of what skills the CASE was 
testing. While it is important to note that our analysis was limited to sample exams, they 
nonetheless provide a meaningful blueprint for the actual exam. For the purposes of our 
analysis, we classified questions testing literary terms and elements as basic knowledge, 
and questions based on the meaning of text were classified as imderstanding/reasoning. 
We then further divided understanding/reasoning questions into the sub-categories of 
understandingD questions based on information stated explicitly in the text-and 
reasoningD questions demanding some level of interpretation or insight into the implied 
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meaning of the text. For the English I sample test, we found that the multiple-choice 
questions were evenly divided between basic knowledge and understanding/reasoning 
questions. We further determined that of the 13 understanding/reasoning questions, 6 
(23%) were understanding questions and 7 (27%) could be categorized as reasoning 
questions. For the English II sample test, there was a slightly larger emphasis on basic 
understanding questions (58% of the total). 1 7% of the total questions were 
understanding questions and 25% were reasoning questions. In both sample exams, 
approximately 25% of the miJtiple-choice questions could be classified as reasoning 
questions. This reveals a far greater emphasis on questions testing students Chbility to 
recognize literary elements and clearly stated textual information as opposed to their 
ability to interpret and evaluate information. 

The constructed response section of the English CASE is intended to test 
students Chbility to identify both explicit and implicit meaning in one of several 
designated core texts. Teachers are given a reading rubric to help grade answers to the 
constructed response section. For the short answer reading rubric, answers are scored on 
a scale of 0 to 4. A score of 2 reflects the 50% mastery necessary to pass the CASE. 
According to the rubric, in order to receive a score of 2, the student must [demonstrate an 
accurate but limited understanding of the text,CI3fiake simplistic interpretations Q Oise 
irrelevant and/or limited references 5 and [generalize without illustrating key ideas □ The 
acceptable baseline for passing, then, has little expectation that the student will be 
capable of much more than a minimal basic understanding of a core text. 

Taken together, these findings suggest that sample CASE exams for English I and 
II do not challenge teachers to emphasize interpretive and implied reasoning skills. 
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While there are surely portions of the sample CASE that require such thinking, it is 
nonetheless possible for a student to pass the English CASE with minimal utilization of 
these skills. In short, the CASE exam reflects the same basic proficiency level as the 
Program of Study. 

IV. TEACHER® PERCEPTIONS: FINDINGS FROM OUR INTERVIEWS 
TeachenUResponses to District Implementation of Standards and Assessments 

In order to assess how teachers responded to both the pressure and support aspects 
of the districts curricular policy as represented by the CASE and Programs of Study, 
respectively, we asked English and mathematics teachers how each of these affected their 
curricular and instructional choices. We coded teachers Cresponses concerning the 
Programs of Study into six categories of use. Goals and objectives indicate that teachers 
used the Program of Study to establish the overall goals for each instructional unit. These 
statements tended to be general; teachers used the Program of Study to guide the overall 
goals for their courses. Literary Work/T opic indicate that teachers used the Program of 
Study to select literary works for their classes or, in math, to decide what math topics 
they should teach. Sequence refers to teachers Duse of the Program to determine the order 
in which they taught units, works or topics. Length indicates that the teachers used the 
Program to decide how many days, weeks, etc. they spent on each unit. Instructional 
Activities refers to reports that teachers used the suggested activities listed in the Program 
of Study. Instructional Approaches refer to teachers Dreports that they used the Program 
of Study to determine how they will actually teach the course materials. This category 
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should indicate that the Program of Study guided how teachers understood course content 
and how it should be taught. 

We used a different scheme to categorize the effects of the CASE that was based 
on the types of responses teachers provided us. We found that most teachers felt that the 
CASE affected their curricular and instructional decisions in four ways. First, teachers 
said that it influenced the Literary works and Topics for study that they selected. Second, 
the CASE affected the Sequence, or order in which teachers presented works and topics 
throughout the semester and the school year. Third, the CASE influenced what Skills the 
teachers focused on in their classes. Finally, teachers reported that the CASE influenced 
their curricular and instructional decisions because they included Test Preparation, or 
lessons focused on preparing students for the format and content of the CASE. 

We found, again, that teachers responded most to the pressure rather than the 
support components, though differences emerged across schools and subject matters. 

As indicated by Table 4, most English teachers (77%) reported that they made some use 
of the Program of Study while less than half of the math teachers (44%) did so. A higher 
percentage of English teachers in Schools C and D, under probation and reconstitution, 
respectively, reported using the Program than in Schools A and B that do not face district 
intervention. English teachers in Schools C and D used the Program of Study to 
determine goals and objectives, what works to teach, when to teach them over the course 
of the semester/year and how long to spend on each unit. The lowest percentage of 
English teachers who reported using the Program of Study were in School B. Only half 
of the English teachers said that they even received the Program of Study, with many of 
them not knowing what they were. In part, this was a result of a change in the 
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department chair. The chair during the study year, 1998-99, taught ESL classes rather 
than English classes. Being new to her role as chair, there were some problems with book 
orders and with dissemination of materials. The low percentage of teachers reporting that 
they received the Program of Study likely reflects this administrative change. 

Significantly, most math teachers in all of the schools reported that they did not 
use the Program of Study. This was true for probation/reconstitution schools as well as 
non-probation schools. Math teachers, when they did use the Program of Study, made 
use of it to select topics rather than to determine goals and objectives or sequencing of 
units. Many math teachers said that the Program of Study was difficult to use. In 
particular, these teachers complained that the Program did not correspond with the 
sequence of instruction laid out in their textbooks and that, in order to implement the 
Program of Study they would have to [jump around Dtoo much. In addition, many math 
teachers resisted the integration of algebra and geometry supported by the Program of 
Study. 

Very few teachers in either subject in the four the schools reported that the 
Programs of Study informed their instructional approaches. English teachers typically 
said that it simply reflected what they already taught, while math teachers typically did 
not even refer to the document. 

In contrast, a higher percentage of both math and English teachers across the 
schools reported that the CASE affected their curricular and instructional decisions. 

While 38% of the teachers reported no effects, 62% of them said that the CASE had some 
effects on their curricular and instructional decisions. As Table 5 indicates, teachers 
reported that the CASE affected what works or topics they taught, when they taught them 
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and that they included more specific test preparation activities than before. Overall, 23% 
of the teachers interviewed said that the CASE influenced the sequence of their 
curriculum. There was no distinct pattern of use by subject matter or school. Interviews 
suggest that for math teachers the effects of the CASE on sequencing were not 
significant These teachers typically reported that they had to make sure they covered 
certain topics during each semester so students would be prepared to take the CASE. For 
English teachers, however, the sequencing changed how they normally approached their 
course material. This was particularly true for teachers teaching tenth grade American 
literature. Many of these teachers reported that they preferred a chronological approach 
to the course. However, because they had to teach A Raisin in the Sun, a contemporary 
work, in the first semester they could not maintain this chronological approach. Given 
the lack of district training on the standards and limited documented rationale for the 
sequencing of the districts English curriculum as detailed in the Program of Study, many 
of the teachers taught the core works discretely. Few drew connections between core 
works over the course of the year. One exception was in School B where teachers did 
begin to relate the works to one another. These teachers had students compare and 
contrast characters and ideas across texts. These assignments began to give a more 
meaningful structure to the curriculum. However, teachers developed this type of 
coherence as they taught and they developed it late in the school year. 

Along with the modest effects of the CASE on sequencing and choice of works or 
topics, teachers reported that the CASE affected their curriculum in that they spent one or 
two weeks before the exam going over sample questions provided by the district. 
However, because the district only provided these sample questions two weeks prior to 
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the administering of the CASE, they did not assist teachers in developing curriculum and 
incorporating standards into their curriculum throughout the year. 

Instructional practices and district standards and assessments 

In order to understand teachers Iltesponses to the pressure and support dimensions 
of the districts curricular policies, we interviewed the eight teachers we observed teach 
To Kill a Mockingbird. The interviews focused on the following questions: 1) How did 
district standards and assessments, the Programs of Study and the CASE influence their 
instructional practices?; 2) How did teachers explain the pressures of the CASE? and 3) 
Did the teachers believe that they had achieved their objectives for the unit? Our 
interview analysis revealed three consistent themes. Coverage, pressure of the CASE and 
time constraints seemed to be important factors in shaping teachers [Iburricular and 
instructional decisions, though the importance of these factors varied by school and by 
teacher. In what follows, we discuss these three factors. 

Coverage 

Teachers deported objectives for the instructional unit were similar across 
schools. Six of the eight teachers said that identifying the elements of a novel was one of 
the main unit objectives. Three of the teachers also noted that they wanted students to 
understand the historical period in which the novel was set, although all mentioned the 
time period throughout the rmit. Teachers in School B and School D explicitly addressed 
historical issues by incorporating activities and readings about the Depression Era into 
their units. Most other teachers simply made references to the Depression and an 
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emerging civil rights movement as these allusions came up in the book. Though teachers 
talked about the importance of the [historical perspectiveQ their approach to teaching this 
perspective was ad hoc and superficial. Teachers typically referred to characters 
throughout the novel as reflections of the social setting of the novel but did not delve into 
how the historical period in which the story was set and in which the author wrote the 
book affected the novel and, in particular, its narration. 

References to identifying literary elements reflect some teachers Qise of the 
Programs of Study. The Programs of Study identifies the central objective of the unit as 
follows: UThe students will read literature and be able to evaluate the relationships 
between a plot and its subplots, connecting themes, character traits, motives, tone, point 
of view and setting.u‘^ Most teachers identified this as their key objective even though 
many of the teachers said that they did not refer to the Programs of Study in their unit 
planning. Only three of the eight teachers said that they used the Programs of Study to 
determine unit objectives. The importance of identifying literary elements reflects a 
conventional way of approaching literature that the majority of English teachers in the 
four schools referred to when they said that the Programs of Study reflected rather than 
challenged how they typically taught 

Though teachers stated the goals of teaching students to identify literary elements 
and to OmderstandOthe novel B historical setting, the predominate objective for all of the 
teachers was to make sure that students read the novel and could identify the story line of 
the novel. Teachers talked about the importance of [getting them (students) to read,Dof 
[getting them to understand the facts of the novel, Dto [just understand the plot. □ 
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Teachers said that most of the students would not read the book on their own and that in 
order to ensure that they at least read the book they needed to read it aloud with them. 

Pressure of the CASE 

Teachers placed importance on simply getting students to read the book in large 
part because of the CASE. One teacher in School A, the school not under district 
intervention, responded when asked how the CASE influenced her lessons; □ just wanted 
to make sure that I covered all the material, that we completed the novel, to arouse their 
curiosity about Boo, make sure diey understood what the title meant in relation to several 
characters in the book. □ The teacher Q comment reflects the concern of all of the teachers 
that students get through the book and, also, that they understand the major themes. For 
all of the teachers, identifying several of the novel sQhemes was a central objective that 
stemmed primarily fiom the CASE. The CASE involves an essay question that requires 
students to discuss one of the themes of a core work. Because teachers did not know 
what theme from Mockingbird that the CASE would focus on, teachers felt that they had 
to cover as many themes as they could. One teacher from School C when asked what the 
objectives of the unit were commented; □ tried to anticipate the CASE essay questions. 
Giving them a thorough understanding of the different themes of the book. □ Teachers 
typically reported that they covered from three to five different themes, including racism 
and prejudice, the idea of a mockingbird, poverty and class structure, empathy, coming of 
age, justice, and appearance versus reality. Teachers felt compelled to cover most of 
these themes because of the CASE. One teacher in School D said, 

□ would think most teachers who are teaching that those would be the 

themes that they would focus on. But the problem could come, though, because 
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for example, with the A Raisin in the Sun CASE question, they asked on the essay 
section, was on a theme that was very vague and was not something I had covered 
and that most of the other teachers had covered. So hopefully they didnU do 
fliatn 

In large part, then, the CASE both structured the curriculum for the teachers and 
fragmented it at the same time. Teachers selected themes based on what they thought 
would be on the CASE. Instead of providing conceptual coherence to the unit, however, 
teachers covered several themes in order to prepare students to answer the CASE 
questions. The pressure of the CASE and, in particular, the fact that the district provided 
a limited curricular rationale and modest guidance on the themes or concepts to be tested, 
contributed to a basic order catalog type of curricular structure in which teachers 
identified several themes without drawing connections between them.'^ Teachers □ 
concern that all students read the entire novel and that they touch on several themes 
reflect teachers Dboncems with the CASE. Only one teacher, from School B, amongst the 
eight observed said that she began to draw connections across texts and that a unifying 
theme had emerged from the core works over the course of the year. Tenth grade 
teachers in School B further focused on these connections by assigning students a 
research paper that had them compare and contrast Atticus from Mockingbird and Mama 
from A Raisin in the Sun in terms of parenting skills. This was the only case of cross-text 
connections that we saw being made in the four schools. 

Time Constraints 

Another concern associated with the CASE was the issue of time. Teachers in 
School C felt constrained by competing pressures that arose from the TAP and probation. 



32 




33 



Challenge of Improving Instruction, Wong, et. al. (2001) 



One teacher in School C said that she did not collaborate with other teachers to develop 

her Mockingbird unit because Ijfcveryone was just running around with the TAP. I didntl 

want to impose on them. □ This teacher reported that she did not assign any writing with 

Mockingbirdhecavtse Q was pressured for time with the TAP and the CASE. I had to 

cram in as many details as possible about the novel. □ The other teacher observed in 

School C, and die special education teacher who worked with him and taught much of the 

Mockingbird unit, also noted that the schools emphasis on the TAP and its relation to 

probation affected how they taught the novel. These teachers initially had planned to 

have students read the screenplay rather than the novel because the majority of the second 

semester had been spent preparing students for the TAP test. These competing pressures 

led these teachers to assign students to read the majority of the novel at home and then to 

have students read the screenplay along with watching the movie rather than completing 

the novel. The head teacher explains his decision; 

Q decided to read the screenplay because of time constraints. I decided to make 
the last two novel tests Mockingbird. Time constraints with the TAP. I ended up 
having only three weeks before the CASE.. .Time. What it was we were just 
concerned about time constraints. Basically weDte teaching reading for the TAP. 
YouSe got two things tugging at you. YouSe got time constraints you have to 
deal with. □ 

The special education teacher who taught with him noted, CWe are not going to sacrifice 
the TAP for the CASE. □ 

Interestingly, the teachers in School D, the school under reconstitution, did not 
mention time constraints like their counterparts in School C. School D was, however, the 
only other school to use the screenplay. Teachers said that this was not a conscious 
decision like in School C, but that it was the result of an ordering mistake. However, the 
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teachers chose to teach the screenplay. For the sanctioned schools, the screenplay served 
both as a way to engage students through oral reading as well as through a simplified 
plot. 

Conclusion 

Teachers in our study clearly felt constrained by the pressures of the CAS and the 
CASE. The standards and assessments served in large measure to regulate curriculum 
planning and instructional choices-particularly for English teachers. Teachers felt the 
pressures of coverage, pressure from the CASE, and time constraints. Teachers in all 
schools discussed how they would have made different curricular and instructional 
decisions if not for district pressures driven by the mandated assessments. In the 
following section, we discuss the impact that the CAS and CASE have had on the nature 
and type of instruction. 

V. TEACHERS ONSTRUCTIONAL PRACTICES 

Along with interviewing teaehers about their eunieular and instruetional 
decisions, we also observed each of the eight teachers teach the novel. We analyzed 
these classroom observations in several ways. Our goal in these analyses was to break 
down classroom instruction to different levels to identify the content and nature of 
teachersDdecisions. In addition, we wanted to see how these decisions affected the kinds 
of instruction and learning opportunities occurring in these classrooms. 

First, we focused on teachers [Instructional decisions. Toward this goal we 
examined teachersDdecisions regarding allocation of instructional time and categorized 
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types of written assignments. Second, we looked more closely at representative samples 
of teacher-led discussions, what we call recitations, in order to understand the nature of 
classroom instructional interactions. To begin with, we identified the questions teachers 
asked students about the novel and analyzed them using a taxonomy of reading skills. 
Then, using this same taxonomy, we reversed the analysis looking at the questions 
students asked teachers. We also use the taxonomy to analyze teachers Qvritten 
assignments. Third, we focused on the dynamic between teachers and students, looking 
at the dialogue initiated by higher-level teacher questions. In this analysis, we wanted to 
imderstand not only how teachers prompted students to answer complex questions, but 
also the nature of student response to these questions. 

Time Allocation and Instructional Activities 

Table 6 indicates the percentage of classroom time teachers allocated to 13 
different types of activities. Classroom transcripts were coded according to the length of 
each episode or activity and the type of activity involved. Using HillockQ coding 
scheme, we considered an episode to be a GbhunkDof time that lasted at least two minutes 
and that was bounded by changes in materials, classroom arrangements and/or 
instructional objectives. Table 7 indicates how we classified activities. 

Along with management activities, teachers allocated most of the instructional 
time observed to four types of activities: recitation, seatwork, reading aloud and watching 
movies and/or listening to tapes of the novel being read aloud. On average, the teachers 
spent 23% of the observed time on recitation, 15% on seatwork, 14% on reading aloud 
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and 10% on watching the movie or listening to an audio-tape of the novel being read 
aloud. These activities accounted for 78% of classroom time. 

Though there are many similarities across classes, there are differences by school. 
Teachers in Schools C and D, those under probation and reconstitution, respectively, 
spent more time on recitation and less time on reading aloud than teachers in School A 
and B. One would expect the opposite given the threat of further district intervention that 
loomed over teachers in Schools C and D. Teachers in these schools should feel more 
compelled to make sure that students read the novel for the CASE. However, the fact 
that these teachers spent more time on recitation meant that they spent more time 
explicating the novel for students. One teacher in School C spent almost the entire unit 
going over a study guide packet for the novel. Students graded each others [jackets as 
the teachers went over each question and the correct answer that was projected on an 
overhead at the front of the class. Teachers in School D frequently interrupted studentsD 
reading aloud to explain what was happening in the novel, to defrne problem vocabulary, 
and to quiz students on the story line or plot. Teachers Ehllocation of time across the 
schools thus reflects the centrality of their stated objective of students [getting the factsD 
of the story and [getting down the plot. □ 

In addition to CASE curriculum, teachers continued to prepare students for the 
TAP exam. Table 8 indicates the amount of time teachers allocated to standardized test 
preparation during the Mockingbird unit. We classifred activities as standardized test 
preparation that engaged students in taking and reviewing sample tests. We also included 
activities mandated by the schools with the specific goal of raising studentsQest scores. 
For example. Teacher 7 in School B began every period with a Daily Oral Language drill. 
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a grammar exercise that the school mandated that teachers included in their lessons. Not 
surprisingly, teachers in Schools C and D, the schools under probation and reconstitution, 
respectively, allocated the highest percentage of observed instructional time to test 
preparation activities. On average, the teachers in School C spent 20% of the time on 
such activities, while teachers in School D spent 37% of their time. The amount of time 
teachers spent on test preparation within as well as across die two schools, however, 
varies significantly. The difference in time allocation between teachers in School D 
reflects, in part, the fact that Teacher 7 taught a regular level tenth grade class, while 
Teacher 8 taught a class intended for college bound tenth grade students. 

Significantly, no time was spent on engaging students in discussion. While 
recitation follows a pattern in which teachers ask a closed-ended question, students 
respond and teachers evaluate the response, discussion entails students and teachers 
constructing interpretations of literature in dialogue with one another. Discussions are 
marked by open-ended questions and exchanges amon^t students, as well as between 
students and the teacher. The fact that no discussion occurred reinforces the teachers □ 
emphasis on students Qiteral understanding of the novel. The lack of discussion is not, 
however, confined to the Chicago Public Schools. Several surveys of English instruction 
in schools nationwide indicate that recitation dominates instructional time and that 
teachers spend almost no time engaging students in discussion.^® 

Analysis of Recitations— Taxonomy 

While the predominance of recitation, seatwork and reading aloud suggests that 
teachers focused their instruction on the literal level, the analysis of time allocation does 
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not preclude the possibility that teachers could be asking interpretive or higher order 
questions. In order to assess the level at which teachers aimed instruction, we analyzed 
three recitations led by each teacher. We only included recitations that lasted at least 5 
minutes. This meant that one teacher in School A only had one recitation. We selected 
recitations that occurred during the beginning, the middle and the end of the instructional 
unit, when possible. We hypothesized that teachers would ask more interpretive 
questions towards the end of the unit as students had become more familiar with the basic 
plot and details of the novel. We categorized questions using Hillocks and LudlowQ 
taxonomy of literary questions.’’ We added four other types of questions that occurred 
frequently within our sample recitations. These are: 1 ) Literary Terms are questions that 
ask students to define or identify literary terms, such as hyperbole, simile, etc; 2) General 
Knowledge Questions that ask students to draw on commonly known facts about people, 
events or objects; 3) Vocabulary Questions that ask students to define specific words 
from the text, and 4) Personal Opinion Questions that asked students to give their own 
opinion about an event or character in the novel. These typically do not ask students to 
provide supporting details or elaboration. 

Table 9 describes what each type of question entails. After coding teacher 
questions, we grouped the questions into three broader categories: 1) Literal (L) 
questions that focus on the basic information stated in the text. These include Basic 
Stated Information, Key Detail and Stated Relationship questions; 2) Simple Inferential 
(SI) questions require readers to make a generalization by connecting typically two pieces 
of information found in close proximity within a literary work. These include Simple 
Implied Relationship questions; 3) Complex Inferential (CL) questions that require readers 
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to connect several pieces of information across a literary text(s) to make conclusions and 
generalizations. These include Complex Implied Relationship, Authors DGeneralization 
and Structural questions. We grouped Literary Terms, General Knowledge, Vocabulary, 
and Personal Opinion into the Other category. 

Our analysis of recitations shows that teachers focused primarily on the literal 
level. Table 10 indicates that the overwhelming majority of questions teachers asked can 
be categorized as Literal. Questions asked at the end of the unit did not differ much from 
those asked at the beginning and middle of the unit. Teachers asked only a few Complex 
Inferential questions, typically at the middle or end of the unit. When teachers asked 
these questions three responses typically occurred; students offered partial answers while 
the teacher answered the question, no response was made, or the teacher indicated that 
the right answer was actually a lower-level, typically Simple Implied or Stated 
Relationship answer. Frequently, teachers asked what appeared to be Authors 
Generalization questions. These questions typically took the form of CWhat is one 
theme of the book?D The notion of theme suggests that teachers wanted students to 
identify what the author was sa 3 dng about an important concept or idea in the novel. 
However, students responded by giving one or two word answers, such as Cfacism,Dor 
[Rowing up. □ Teachers accepted these answers as sufficient and seldom asked for or 
provided elaborations. We coded these questions as Simple Implied Relationships 
because the teacher did not seem to expect elaboration and thus reduced the complexity 
of the answer. In order to explain an Authors Generalization one would have to 
elaborate on what the author is saying about racism or about growing up. Simply saying 



39 



O 

ERIC 



40 



Challenge of Improving Instruction, Wong, et. al. (2001) 



that these are important does require the student to make some implicit generalizations 
but it does not require the student to provide a meaningful interpretation of the work. 

On the whole, then, teachers Oquestions indicate the importance teachers placed 
on getting students to identify the Cfacts of the book. □ Teachers seldom asked students to 
make complex inferences about the characters or relationships in the novel, nor did they 
ask students to make elaborated generalizations about the novels themes. 

Our analysis of both the types of teachers Dwritten assignments and the types of 
questions asked reflects the same focus on plot mastery. A little over half of the teachers □ 
written assigmnents took the form of worksheets with true/false, multiple choice, and 
short answer questions. The remaining written assignments took the form of writing 
exercises, vocabulary exercises, and tests or quizzes. We found that 65% of the questions 
on the written assignments were literal, and a third were inferential. Table 1 1 indicates 
the classroom assignments by type. Table 12 illustrates the types of questions on written 
assignments. 

Teacher-Student Interactions 

What is the experience of students in the CASE classroom? We examined this 
question in two ways. First, we looked at student questions using the same taxonomy 
used with teacher questions. Second, we focused on the dialogue segments proceeding 
inferential questions. Our objective here was twofold. First, based on our earlier 
findings that teachers devoted the majority of their instructional time to literal 
comprehension, we wanted to know if we could discern from the data if teachers IDwere 
reacting to the sense that this was the level of student knowledge. We believed that by 
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looking at student questions, we could gauge, to a certain degree, the level of student 
engagement. Second, we wanted to understand how teachers negotiated inferential 
questions. What kinds of answers did they consider adequate? Did they model the type 
of answers they expected? 

Taxonomy of Student Questions 

In order to assess the kinds of questions students asked, we coded student 
questions into five categories based on Hillock Q taxonomy and our additional categories. 
Literal questions and inferential questions retain their earlier definitions. In addition, we 
added procedural questions and off-task questions. We classified procedural questions 
as those in which students asked about the nature of the classroom activity, such as; What 
question are we on? What are the directions? We classified questions as off-task when 
students changed the subject or discussed something unrelated to the topic initiated by the 
teacher. 

In the classrooms we observed, we were surprised to find that just over half of 
students Dquestions focused on procedural type questions. Table 13 shows the types of 
student questions across instructional units. While we did observe a large degree of 
variation between teachers □ certain activities generated more procedural questions than 
othersD except for one teacher, this variation was between 40 and 75%. (While we found 
this surprising, it is not within the scope of this report to account for this.) After 
procedural questions, one-fourth of the questions students asked were literal. Eleven 
percent were inferential. In other words, approximately a third of all questions were 
substantive questions about the text. 
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Teacher prompting 

The type of student engagement in the classroom, however, is often a reflection of 
the goals of the individual teacher. Teachers demonstrate their expectations from 
students by encouraging or modeling certain responses. Students learn by example as 
well as through prompting. In an effort to understand the dynamic between teachers and 
students, and specifically how teachers negotiated inferential questions in our CASE 
study classrooms, we developed an approach that we felt would elucidate this issue. 

First, we identified the full instructional moment following an inferential question. An 
instructional moment began when the question was asked and ended when the topic of 
conversation moved to another topic. Then we categorized the types of questions or 
prompts teachers asked. Individual prompts occurred when a teacher asked an individual 
student to follow up on a previous statement. Whole class prompts occurred when the 
teacher directed a follow-up question to the entire class. We also looked at the kind of 
answers teachers provided for students and students provided for teachers. We 
categorized these as simple statements, when teachers or students gave an answer that 
was either one word or a sentence, or as substantiated statements, when they provided an 
answer that was supported with evidence from the text. In our estimation, when teachers 
and especially students made substantiated statements, they demonstrated knowledge of 
the novel as well as the ability to apply direct evidence to a statement. This represented a 
more complex response than one when students show they know the answer but fail to 
develop it. It is important to clarify that by definition, an inferential question is complex, 
thus necessitating an elaborated response to be answered in full. We present our findings 
through two different lenses. Tables 14 and 15 provide a summary analysis of 
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instructional moments following simple and complex inferential-type questions. Tables 
16 and 17 look at this same data on a teacher-by-teacher basis. 

In our analysis of instructional moments, we found that there was not a lot of 
variety among the various teachers in our study. Teachers rarely abandoned an 
inferential question after one student answer. Rather, teachers tended to persist on the 
question until they felt that the question had been fully explored. Of teacher prompts, the 
majority focused on having an individual student clarify his or her simple answer. 
However, the teachers did prompt the entire class a third of the time. We also found 
teachers did model substantiated statements □ with fifty percent of their statements 
substantiated. In addition, our instructional moments tended to follow a similar pattern. 
First, the teacher would ask the inferential question. Then students would provide their 
responses. When the students did not provide the answer expected, the teacher would 
prompt either an individual student or the entire class. Finally, once satisfied that the 
answer had been fully explored, the teacher would conclude the instructional moment by 
providing the full answer she had wanted. While often this was a summary of what 
students had contributed, for teachers it served the purpose of relaying to students the 
answer to the question. We observed this pattern over and over again. In none of our 
analyses did we find a student providing this summary. 

What does our analysis of student contributions reveal about why teachers 
ultimately provided the answer to the question themselves? In our sample instructional 
moments, student only substantiated their statements 13% of the time. Teachers, while 
modeling the kinds of responses they wanted, did not expect this type of response from 
their students. As a result, teachers felt compelled to do much of the work themselves. 
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This leads us to conclude that teachers could be pressing students more, being more 
explicit in having students substantiate answers. Our findings indicate that while teachers 
may be engaging in substantiation, they are not explicitly encouraging students to do so. 
This finding is encouraging as it indicates that teachers are conveying complex ideas to 
students. At the same time, however, it indicates that teachers need to prompt students to 
generate more of the complex answers themselves. 

Conclusion 

Efforts to implement standards-based instruction and assessments are t)^)ically 
associated with two goals: accountability and improving instruction. In this study, we 
found that teachers responded primarily to the accountability dimension of the districts 
curricular policies rather than to the instructional improvement aspect. Teachers reported 
that the districts assessment, the CASE, did influence their curricular and instructional 
decisions. The main effect of the CASE was on sequencing of topics or works. In 
English, teachers reported that the CASE made it impossible to use a chronological 
approach to teaching American Literature. This did, then, result in altering the structure 
of the English curriculum. Rather than providing a thematic or conceptual structure, 
however, the lack of an explicit district rationale for the sequence of works implied by the 
CASE, resulted in Augmenting the curriculum. Some teachers did draw out common 
themes across the core works, but these were done on an individual or school-wide basis 
rather than on a district-wide basis. Further, these connections were made late in the 
school year. Any coherence teachers created was, then, ad hoc rather than planned. 
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The CASE also affected the sequence of math instruction, though not as 
significantly. Math teachers reported that they made sure they covered the topics that 
would be on each semesters CASE. Math teachers did not have to reconceptualize their 
curriculum. Interestingly, the Program of Study for math did attempt such a 
reconceptualization by integrating geometry and algebra. Teachers felt that this required 
them to I^ump aroundDtoo much within their textbooks. In response, few math teachers 
said they used the Program of Study, the districts primaiy support mechanism, to design 
their curriculum. 

English teachers in the sample schools tended to use the Programs of Study more 
than math teachers. English teachers used the Program to identify objectives, core works, 
and sequence. However, it is doubtfiil that this actually changed how they taught. The 
overwhelming number of English teachers said that the Programs of Study had no effect 
on their instruction. Many teachers said that it simply reflected what they had been doing 
for years. 

Teachers [responses to the accoimtability rather than instructional aspect of the 
districts policy were due, in large part, to the way in which the district itself 
implemented the policy. During the early phase of policy implementation, the district 
provided few supports to teachers or schools. Very few teachers in the four schools 
reported attending district training on the rationale of the standards and assessments, and 
on the instructional theory that lies behind them. Further, a close analysis of the 
supportive materials the district did provide English teachers suggests that the primary 
intent of the standards and assessments was to standardize rather than improve 
instruction. The English Program of Study represents a conventional view of English 
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instruction focused on the different types of writing and on identification of the various 
elements of each type. Though the districts standards include mention of historical and 
cultural perspectives and language associated with constructivist teaching, the Programs 
of Study contains few references to such perspectives, stressing instead a lower order 
type of literary analysis focused on identifying literary elements. 

Our analysis of how teachers taught one of the districts mandated core works. To 
Kill a Mockingbird, indicates again how the teachers responded more to the 
accoimtability rather than instructional aspects of the districts curricular policy. The 
primary goal of the imit was to cover the novel. Teachers wanted to make sure that 
students read the novel and that they could identify the plot, characters and many of die 
novels themes. The importance of these goals is reflected in the teacher interviews and 
in the classroom observations. Teachers Dquestions focused on the basic information of 
the novel. Even when teachers asked apparently more interpretive questions about the 
novels theme, the accepted answers were typically one or two word phrases. In essence, 
teachers viewed the themes as just one more CfactDof the novel, not ideas or concepts to 
be constructed through reading and discussing the novel. This idea of what constitutes a 
theme reflects efforts to cover the novel and to make sure that students were prepared to 
take die CASE. Again, die teachers are responding to die accountability aspect of the 
districts curricular policy. The CASE becomes the driving force behind the districts 
curricular policy. Teachers repeatedly said that they focused on the themes that they 
thought would be on the CASE. Because teachers felt that they needed to touch on as 
many themes as they could, the districts policy reinforces, if not results in, a fragmentary 
or catalog approach to curriculum. 
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VI. CONCLUSIONS 

Summary of Findings and Policy Recommendations 

Overall, our four-school study reveals the challenge of improving instruction 
system-wide in urban districts such as Chicago. Our study highlights these difficulties as 
it examines the early implementation of Chicago H standards and assessments. The fact 
that we found little significant changes in teachers [instructional practices reflects both 
the early implementation stage and the challenges of system-wide instructional change. 
Though our study lacks a baseline by which to compare teachers Eiurricular and 
instructional practices, teachers report that they have not changed their instruction in 
response to the standards and assessments. The few changes that occurred were not 
significant; teachers report that they re-sequenced some topics and that they took time out 
of the curriculum to prepare students for the district assessment through the use of sample 
questions and tests. Our analysis of the district documents intended to support teachers □ 
efforts to teach to the standards suggests that the Programs of Study has placed primary 
emphasis on basic competency. The English Program of Study tends to reduce the 
complexity of the districts standards and supports a conventional, basic literal-oriented 
instruction. Since we focus here primarily on English instruction, we see a need for more 
research on how teachers in other subject matter responded to the standards. 

While efforts to implement standards and assessments are often assumed to be 
attempts to improve instruction, our study suggests that in the short-run urban districts 
may be more successful in using these tools to standardize curriculum. Improving 
instruction, however, is a longer-term challenge and is likely to require other policy 
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support mechanisms at the district level. Our study indicates the need to consider how a 
district balances the accountability and instructional dimensions of standards and 
assessments and the nature of the support the district provides teachers. Further, we need 
to consider the nature of the standards and assessments themselves. Having standards 
says very little about the quality of the standards or about their various uses. Finally, 
ChicagoS standards-based efforts need to be considered within the context of the nature 
of the challenge in improving urban high schools. Clearly, the standards provide a base 
line or minimum level of instructional quality in a system with a high number of lower 
performing high schools. Further research is needed to develop a more systematic 
understanding of how district-wide standards and assessments are shaping instructional 
and curricular practices in low performing high schools. Standards and assessments 
constitute an important first step, albeit inadequate as a self-contained strategy, toward 
instructional improvement in large urban systems. 

This study also suggests that system-wide standards and assessments have had 
some impact on regulating teachersDcurricular and instructional choices. The district is 
to be commended for developing a content-based exam. But while the district may be 
seeing improvement in test scores, greater effort is required to improve teachers □ 
curricular decisions and instructional interactions with students. While some of this 
uniformity may change as teachers are given the discretion to choose the core novel, as 
expected in the 2000-2001 CASE, it still does not address the teachers Doverall concerns 
regarding coverage, pressure, and time, as they prepare students for the assessment. The 
district needs to decide if the CASE is intended as a test to set basic proficiency levels, or 
as a lever for improving student achievement. The district, in conjunction with school 
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administrators and teachers, needs to work to provide instruction appropriate for the 
student population at each particular school site. 

Balancing Regulation, Support, and Professional Discretion 

Over the last four years, the districts educational agenda has centered on the use 
of regulation, support, and personal discretion. A key challenge for the district has been 
to strike a balance between these different policies. Through the CAS and the CASE, the 
district has provided teachers with clear regulation. At the same time, our study suggests 
that the district could provide more sujjport for teachers. Perhaps it is as a consequence 
of the districts effort to allow for professional discretion that they have provided very 
little professional development linked to the CAS or the CASE. Nor have they shifted 
such professional development expectations to the school level. We suggest that the 
district work with schools to provide targeted instructional support to teachers in order to 
offer them a greater repertoire of activities to turn to as they prepare students for district- 
wide exams. 

Indeed, our study has shown that by and large teachers have the skills necessary 
to move towards a higher level of instruction. The teachers in our study are prepared, 
have a strong imderstanding of the core works they teach, and are working hard to 
prepare their students for the CASE. The question that remains is how teachers can be 
supported to make better decisions in their classrooms. For example, how can teachers 
raise their level of questioning so that students are more challenged by inferential 
questions? How do we get students to be more engaged in the material? The district 
needs to think of strategies to add to the existing capacities of its teachers. Possible 
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support mechanisms might include initiatives ranging from sustained professional 
development at the districts newly established National Teachers CAcademy to a wider 
array of resources for teachers posted on the Chicago Public Schools website. 

In addition, the district needs to decide whether its standards and assessments are 
meant to serve a basic proficiency purpose or should serve as levers to improve the 
quality of instruction. This study suggests that thus far the district has focused primarily 
on the former agenda. We believe the district should be more ambitious in its 
instructional agenda. Higher-order learning skills and constructivist approaches to 
teaching are clearly imbedded in the CAS. They are, however, far less evident in the 
Programs of Study and the CASE. By re-calibrating the Programs of Study and the 
CASE to ensure that higher-level skills are not displaced in the tunneling process, 
teachers would be challenged to pursue the more innovative as well as the more 
traditional aspects to the Chicago standards and assessments. 

As currently structured, district support does not account for variation between 
schools and within schools. Given the wide range of abilities of students in the CPS, 
each individual school needs to assess how the standards and assessments can be used 
best with their students. Instruction in International Baccalaureate or Advanced 
Placement classes, for example, should be different than instruction for students 
struggling with basic literacy. At this point, there are no mechanisms in place to help 
teachers assess what is appropriate for their students and as a consequence they appear to 
be making very similar instructional decisions. While these are clearly school-level 
challenges, the district needs to provide support to schools to facilitate this processes. 
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The Role of School-level Support 

Our findings indicate that teachers develop an instructional approach based on 
their assessment of vk'hat will be tested on the district-wide CASE. This process occurs 
with little direct support from either the district or individual schools. Our study suggests 
that there are school-level mechanisms that could better assist teachers in their 
preparation of students for the CASE. We believe that subject matter departments are 
currently underutilized in just such a role. The subject matter field plays a role in 
shaping teachers attitudes about the technology of teaching, as well as providing a 
rationale for what material should be presented as well as appropriate instructional 
activities. Subject matter departments also determine teachers Dnetworks within the 
school, shaping, in large part, their support systems as well as their collegial 
interactions.'* Departments promote professional exchange, help teachers in their 
decision-making processes, and enhance teachers Cknowledge of their subject matter. In 
short, they shape teachers Cbommon norms of practice. And yet our study found that 
teachers often did not turn to their departments for help other than in deciding which 
book to teach. We suggest that each school examine the role of its department in 
supporting teachers as they prepare for the CASE. In addition, we recommend that 
departments become more involved in monitoring instruction. This does not need to be 
an evaluative process, and indeed under current regulations only the principal and 
assistant principal can evaluate teachers. Rather, the department needs to be organized 
around good practices and needs to foster an environment of support for its teachers. 

With the standards and assessments organized by subject matter, it seems natural to 
expect departments to provide support to teachers. 
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Second, schools should work to strengthen professional networks within and 
beyond the school. Teachers across subject matters share students and instructional 
techniques. Drawing on the strengths of the school faculty, the school can highlight 
teachers Qndividual talents, especially when colleagues are asked to participate in 
professional development 

Linkages to Higher Education Institutions 

If the CAS, the POS and the CASE are the primaiy entities structuring curriculum 
and instruction in the Chicago Public Schools, then pre-service teachers should be well- 
versed in these documents before they begin formal instruction. To achieve this, the 
district should improve linkages with local teacher preparation programs to ensure that 
they prepare their students for district expectations. In particular, pre-service teachers 
need to be exposed to the districts standards and assessments which serve as the basis for 
all subsequent academic goals. Teachers who arrive well-versed in the wide-ranging 
goals of the CAS will be better equipped to find ways to implement these goals in their 
curricular and instructional expectations. 

Local universities can also be called upon to help provide more support to 
teachers in the form of professional development. This might involve offering teachers 
the opportunity to audit graduate-level courses in history, math and literature that would 
enhance teachersCknowledge base, as well as enabling an on-going connection between 
secondary and post-secondary level learning. To our knowledge, local universities have 
not been called on to assist in this capacity. 
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To conclude, our intention in this report was to provide a multi-level analysis of 
the implementation of the districtQ standards. Programs of Study, and CASE exam. Our 
findings suggest that the district has succeeded in regulating teachers Dvork and 
structuring the curriculum across schools. While teachers are prepared and have a strong 
foundation, the district still faces challenges. Our report suggests that several issues need 
further consideration. First, the district needs to decide if its standards and assessments 
are intended to serve a basic proficiency purpose or if they are to raise the instructional 
bar. If its goal is to raise performance, the district needs to provide greater support to 
schools and teachers in the form of professional development and instructional support. 
Teachers need to be helped both in developing strategies to motivate students and in 
assessing their students Qnstructional needs. They also need to be encouraged to spend 
more class time teaching students to respond effectively to inferential-type questions. 

Further, several organizational supports can be strengthened. First, the district 
can work with schools to expand the role of the subject matter department. Second, 
collegial networks within schools should be strengthened. Finally, the district may turn 
to local universities for support in both the pre-service and professional development 
arenas. 

As standards and assessments reconfigure the work of teachers, ongoing research 
is needed to assess the quality and effectiveness of the support given to teachers by the 
district and schools. In addition, more research needs to be done on the effect of across- 
the-board standardization. If teachers are not meeting their diverse student needs and are 
instead making sure that basic proficiency is met, new policy mechanisms must be 
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designed to provide a proper balance between curriculum standards and differential 
academic needs. 
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Type of Support* 



School 


Subject 


No 

Support 


Some 

Support 


Sample Exam 
& Questions 


Program of 
Study 


staff 

Development 


A 


English 

(N=11) 


0.0% 


100.0% 


55.0% 


100.0% 


9.0% 




Math 

(N=9) 


0.0% 


100.0% 


66.0% 


67.0% 


11.0% 




Total 


0.0% 


100.0% 


55.0% 


95.0% 


10.0% 


B 


English 

(N=10) 


10.0% 


90.0% 


50.0% 


70.0% 


10.0% 




Math 

(N=12) 


0.0% 


100.0% 


67.0% 


50.0% 


0.0% 




Total 


5.0% 


95.0% 


59.0% 


59.0% 


5.0% 


c 


English 

(N=7) 


0.0% 


100.0% 


43.0% 


100.0% 


0.0% 




Math 

(N=6) 


0.0% 


100.0% 


89.0% 


100.0% 


11.0% 




Total 


0.0% 


100,0% 


69.0% 


100.0% 


6.0% 


D 


English 

(N=7) 


0.0% 


100.0% 


43.0% 


100.0% 


0.0% 




Math 

(N=6) 


0.0% 


100.0% 


67.0% 


83.0% 


0.0% 




Total 


0.0% 


100.0% 


54.0% 


92.0% 


0.0% 


Total 


English 

(N=35) 


3.0% 


97.0% 


49.0% 


89.0% 


6.0% 




Math 

(N=36) 


0.0% 


100.0% 


69.0% 


78.0% 


3.0% 


All 

Teachers 


(N=71) 


1.0% 


99.0% 


59.0% 


83.0% 


4.0% 



* NOTE: Sample exam & questions: Teachers report receiving sample CASE questions 
and practice exams from the Central Office; Programs of Study. Teachers report having 
individual copy of Program of Study for courses; Staff development: Teachers report 
attending Central Office provided in-services directly concerned with CASE exam or 
Programs of Study 
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School 


Subject 


No 

Use 


Some 

Use 


Goals & Lit. 

Objectives Works, 
Topics 


Type of Use 

Sequence Length of Instructional 
Instruct Activities 
Units 


Instnictional 

Approaches 


A 


English 

(N=11) 


27.0% 


73.0% 


45.0% 


55.0% 


9.0% 


9.0% 


9.0% 


9.0% 




Math 

(N=9) 


44.0% 


56.0% 


22.0% 


33.0% 


0.0% 


0.0% 


22.0% 


0.0% 




Total 


35.0% 


65.0% 


35.0% 


45.0% 


5.0% 


5.0% 


15.0% 


5.0% 


B 


English 

(N=10) 


50.0% 


50.0% 


10.0% 


30.0% 


10.0% 


10.0% 


0.0% 


0.0% 




Math 

(N=12) 


67.0% 


33.0% 


25.0% 


25.0% 


0.0% 


0.0% 


0.0% 


0.0% 




Total 


59.0% 


41.0% 


18.0% 


27.0% 


5.0% 


5.0% 


0.0% 


0.0% 


C 


English 

(N=7) 


0.0% 


100.0% 


71.0% 


71.0% 


43.0% 


43.0% 


14.0% 


0.0% 




Math 

(N=9) 


44.0% 


56.0% 


22.0% 


56.0% 


0.0% 


0.0% 


0.0% 


0.0% 




Total 


25.0% 


75.0% 


44.0% 


63.0% 


19.0% 


19.0% 


6.0% 


0.0% 


D 


English 

(N=7) 


0.0% 


100.0% 


71.0% 


86.0% 


57.0% 


43.0% 


0.0% 


14.0% 




Math 

(N=6) 


67.0% 


33.0% 


17.0% 


33.0% 


17.0% 


17.0% 


0.0% 


0.0% 




Total 


31.0% 


69.0% 


46.0% 


15.0% 


8.0% 


8.0% 


0.0% 


0.0% 


Total 


English 

(N=35) 


23.0% 


77.0% 


46.0% 


57.0% 


26.0% 


23.0% 


6.0% 


6.0% 




Math 

(N=36) 


56.0% 


44.0% 


22.0% 


36.0% 


3.0% 


3.0% 


6.0% 


0.0% 


At! 

Teachers 


(N=71) 


39.0% 


61.0% 


34.0% 


46.0% 


14.0% 


13.0% 


6.0% 


3.0% 
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School Subject 


No Effects Some Effects 


Lit. 

Works, 

Topics 


Sequence 


Skills 


Test 

Preparation 


A 


English 

(N=11) 


55.0% 


45.0% 


36.0% 


9.0% 


9.0% 


18.0% 




Math 

(N=9) 


33.0% 


67.0% 


22.0% 


22.0% 


0.0% 


22.0% 




Total 


45.0% 


55.0% 


30.0% 


15.0% 


5.0% 


20.0% 


B 


English 

(N=10) 


10.0% 


90.0% 


40.0% 


40.0% 


10.0% 


10.0% 




Math 

(N=12) 


50.0% 


50.0% 


25.0% 


33.0% 


0.0% 


8.0% 




Total 


32.0% 


68.0% 


32.0% 


36.0% 


5.0% 


9.0% 


C 


English 

(N=7) 


29.0% 


71.0% 


14.0% 


43.0% 


14.0% 


0.0% 




Math 

(N=9) 


67.0% 


33.0% 


0.0% 


0.0% 


11.0% 


22.0% 




Total 


50.0% 


50.0% 


6.0% 


19.0% 


13.0% 


13.0% 


D 


English 

(N=7) 


14.0% 


86.0% 


57.0% 


14.0% 


0.0% 


14.0% 




Math 

(N=6) 


33.0% 


67.0% 


17.0% 


17.0% 


0.0% 


33.0% 




Total 


23.0% 


77.0% 


38.0% 


15.0% 


0.0% 


23.0% 


Total 


(N=71) 


38.0% 


62.0% 


27.0% 


23.0% 


6.0% 


15.0% 
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School Teacher Minutes Spent on 
Standardized Test 
Preparation 


Number 

of 

Minutes 

Observed 


Percent of Time 
Spent on 

Standardized Test 
Preparation 


A 


1 


0 


539 


0.0% 




2 


0 


367 


0.0% 




Total 


0 


906 


0.0% 


B 


3 


0 


434 


0.0% 




4 


50 


383 


13.0% 




Total 


50 


817 


6.0% 


C 


5 


50 


376 


13.0% 




6 


91 


346 


26.0% 




Total 


141 


722 


20.0% 


D 


7 


414 


752 


55.0% 




8 


39 


485 


8.0% 




Total 


453 


1237 


37.0% 


Total 


8 


644 


3682 


17.0% 
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Management 

During management episodes teachers discuss issues relevant to the running of 
the classroom and/or a specific activity. Management activities include teachers 
giving instructions, talking about students[;grades, the class agenda for the day 
or for future days, placing students into small groups, disciplinary issues, and 
taking attendance and record-keeping. 

Discussion 

Discussion involves students elaborating upon ideas or interpretations of an 
individual text, across texts, or between texts and the larger world. It has two 
distinguishing features; up-take and authentic questions. Up-take occurs when 
teachers incorporate student responses into subsequent questions, and when 
teachers and students elaborate upon student ideas and responses. Authentic 
questions do not have pre-specified answers. They are open-ended questions 
that students have to provide evidence or support to answer. Both uptake and 
authentic questions have to be present at the same time for talk to be considered 
discussion. 

Groups 

Students work with one or more other students to engage in and/or complete an 
activity. We did not distinguish between cooperative learning and collaborative 
seatwork. 

Student Report 

Students present material, including their own writing, to the class. 

Recitation 

Recitation typically involves a pattern of the teacher asking a question, students 
providing a response, and the teacher either evaluating the response or simply 
taking the response and moving on to the next topic. Recitation is marked by 
questions that the teacher believes have a right or wrong answer. Recitation is 
also marked by teachers moving from one topic to another, or from one question 
to another question unconnected to the previous question. 
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Lecture 

Teachers provide students with information or explain something without student 
comment. 



Seatwork 

Students work independently, typically with a worksheet. 

Reading Aloud 

The teacher reads aloud to students, students read aloud in turns or parts, and 
students read aloud in small groups. 

Silent Reading 

Students read assigned work by themselves. 

Movie / Audio 

Students watch a movie or videotape, or they read as they listen to an audio 
taped version of a literary work. 

Test 

Students take teacher-created quiz or test in class 
Standardized Test 

Students take practice tests that simulate the content or format of standardized 
tests. 

Diversion 

This includes time spent discussing issues not relevant to understanding the 
course content, time when a teacher is not in the room, and time when nothing 
instructional or management-oriented is occurring. 
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Basic State Information 

These questions ask students to identify information that is directly stated in a literary 
work. Hillocks and Ludlow (1984) define these questions as [basicDbecause they have 
to do with conditions of central importance to a story. Many of the questions teachers 
asked in our sample were not basic in this sense. We coded them as Basic Stated 
Information because this is the lowest category of questions included in HillocksD 
taxonomy. 

Key Detail 

These questions focus on information that is crucial to the plot. Key details typically 
occur at important junctures of the plot and have some causal relationship to what 
happens. 

Stated Relationship 

These questions require the reader to identify the relationship that exists between two or 
more pieces of information. The relationship is stated in the text, typically in the form of 
a [becauseDstatement. 

Simple Implied Relationship 

These questions require the reader to make an inference about a relationship within a 
literary work by drawing on two or more pieces of information closely located within the 
work. 

Complex Implied Relationship 

These questions require readers to make inferences based on several pieces of 
information located across a literary work. They involve a large number of details that 
must be dealt with together. 

Authors Generalization 

This type of question asks readers to identify and elaborate upon an authors view of the 
human condition. They differ from complex implied relationship questions because they 
require the reader to go beyond a specific relationship within a literary work to make a 
proposition about what that relationship implies about human nature as it exists outside 
of the literary work. 

Structural Generalization 

These questions require the reader to explain how parts of a literary work operate 
together to create certain effects. To answer these questions, the reader must first 
generalize about the arrangement of certain parts of a work and then explain how these 
parts work together to create an effect. 

Source: Hillocks, G. and Ludlow, L. (1984). A taxonomy of skills in reading and 
interpreting fiction. American Educational Research Journal, 21:1, pp. 7-24. 
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School 


Teacher 




L 


SI 


Cl 


0 


Total 


A 


1 


N 


35 


2 


0 


2 


39 






Percent % 


90 


5 


0 


5 


100 


A 


2 


N 


21 


2 


2 


0 


25 






Percent % 


84 


8 


8 


0 


100 


B 


3 


N 


51 


2 


0 


0 


53 






Percent % 


96 


4 


0 


0 


100 


B 


4 


N 


58 


18 


8 


9 


93 






Percent % 


62 


19 


9 


10 


100 


C 


5 


N 


26 


4 


3 


9 


42 






Percent % 


62 


10 


7 


21 


100 


C 


6 


N 


34 


18 


5 


5 


62 






Percent % 


55 


29 


8 


8 


100 


D 


7 


N 


43 


7 


1 


6 


57 






Percent % 


75 


12 


2 


11 


100 


D 


8 


N 


54 


30 


1 


4 


89 






Percent % 


61 


34 


1 


4 


100 




Total 


N 


322 


83 


20 


35 


460 






Percent % 


70 


18 


4 


8 


100 
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School 


Teacher 


Worksheets Writing 


Vocabulary 


Test/Quiz 


Other 


Total 


A 


1 


5 


0 


5 


0 


0 


10 


A 


2 


7 


2 


2 


0 


0 


11 


B 


3 


12 


2 


1 


5 


2 


22 


B 


4 


18 


7 


1 


1 


3 


30 


C 


5 


4 


1 


2 


0 


1 


8 


C 


6 


10 


3 


2 


2 


1 


18 


D 


7* 


NA 


NA 


NA 


NA 


NA 


NA 


D 


8 


1 


1 


0 


1 


1 


4 


Total# 




57 


16 


13 


9 


8 


103 


% of ToUi! 


53 


19 


13 


8 


7 




* NOTE: Teacher seven did not assign written work during the classes we observed. 




66 



70 



Challenge of Improving Instruction, Wong, et, al. (2001) 



School 


Teacher 




Literal 


SI 


Cl 


Other 


A 


1 


N 


58 


23 


6 


3 






Percent 


64 


26 


7 


3 


A 


2 


N 


39 


15 


9 


3 






Percent 


59 


28 


13 


0.5 


B 


3 


N 


151 


14 


6 


0 






Percent 


87 


8 


3 


0 


B 


4 


N 


189 


13 


8 


1 






Percent 


85 


6 


3.5 


5 


C 


5 


N 


111 


40 


17 


3 






Percent 


65 


23 


10 


1.5 


C 


6 


N 


87 


22 


11 


2 






Percent 


70 


18 


10 


2 


D 


T 


N 


NA 


NA 


NA 


NA 






Percent 










D 


8 


N 


21 


3 


7 


0 






Percent 


66 


9 


22 


0 


Total: 




N 


656 


130 


62 


25 






Percent 


75 


15 


7 


3.5 






Literal: 


65% 












Inferential: 


31% 












Other 


4% 









* NOTE: Teacher seven did not assign written work during the classes we 
observed. 





Challenge of Improving Instruction, Wong, et. al. (2001) 




School Teacher 


L 


SI 


Cl 


0 


P 


0-T 


Total 


A 


1 


N 


3 


3 


0 


1 


11 


1 


19 






Percent 


16 


16 


0 


5 


58 


5 


100 


A 


2 


N 


7 


3 


2 


2 


3 


1 


18 






Percent 


39 


17 


11 


11 


17 


5 


100 


B 


3 


N 


6 


0 


1 


1 


20 


0 


28 






Percent 


21 


0 


4 


4 


71 


0 


100 


B 


4 


N 


5 


0 


0 


1 


4 


0 


10 






Percent 


50 


0 


0 


10 


40 


0 


100 


C 


5 


N 


2 


0 


1 


1 


12 


0 


16 






Percent 


13 


0 


6 


6 


75 


0 


100 


C 


6 


N 


5 


2 


1 


0 


12 


3 


23 






Percent 


22 


9 


4 


0 


52 


13 


100 


D 


7 


N 


5 


0 


2 


3 


10 


4 


24 






Percent 


20.5 


0 


8 


12.5 


42 


17 


100 


D 


8 


N 


5 


2 


1 


2 


18 


2 


30 






Percent 


16 


7 


3 


7 


60 


7 


100 


Total 




N 


38 


10 


8 


11 


90 


11 


460 






Percent 


23 


6 


5 


6 


54 


6 


100 
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student Contributions 


Teacher Contributions 


Total Contributions 


Type of 
Interactions 


% 


Type of 
Interactions 


% 


Type of 
Interactions 


% 


Substantiated 

statements 


11 


Substantiated 

statements 


30 


Substantiated 

statements 


20 


Simple 

statements 


76 


Simple statements 


23 


Simple statements 


52 


Clarifying 

questions 


3 






Clarifying questions 


2 


Procedural 

questions 


4 


Procedural 

statements 


9 


Procedural 


6 


Off-task 


6 






Off-task 


3 






Individual prompts 


23 


Individual prompts 


10 






Whole class 
prompts 


15 


Whole class 
prompts 


7 


n=182 




<o 

II 

c 




n=328 
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student Contributions 


Teacher Contributions 


Total Contributions 


Type of 
Interactions 


% 


Type of 
Interactions 


% 


Type of 
Interactions 


% 


Substantiated 

statements 


11 


Substantiated 

statements 


38 


Substantiated 

statements 


23 


Simple 

statements 


72 


Simple statements 


6 


Simple statements 


41 


Clarifying 

questions 


0 






Clarifying questions 


0 


Procedural 

questions 


15 


Procedural 

statements 


17 


Procedural 


16 


Off-task 


2 






Off-task 


1 






Individual prompts 


29 


Individual prompts 


14 






Whole class 
prompts 


10 


Whole class 
prompts 


5 


n = 54 


100 


00 

II 

c 


100 


n = 102 


100 
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Total 

teractions 


NA 


8 

100 


3 

100 


41 

0.11 


6 

100 




14 

100 


16 

100 


88 


S 




















Whole- 

class 

Prompts 


NA 


1 

12.5 


o o 


4 

10 


o o 


o o 


o o 


o o 


IT) lO 


It 




















li 

■D c 
c Q. 


NA 


o o 


o o 


8 

20 


o o 


2 

14 


4 

28 


o o 




11 
■a 1 

if 


NA 


1 

12.5 


CO 


•«- CN 


o o 


o o 


•<- 00 


4 

25 


00 00 


0. CO 




















Simple 

Statements 


NA 


o o 


o o 


T- CM 


o o 


2 

14 


o o 


o o 


CO CO 


■o 

£ jS 
« c 
^ o 

II 


NA 


2 

25 


o o 


5 

12 


2 

33 


2 

14 


CO f;; 


4 

25 


CO 

'T* "T— 


ll 

CO 




















Off- 

task 


NA 


1 

12.5 


o o 


o o 


o o 


o o 


o o 


o o 




■S ® 

si 


NA 


o o 


CO 


o o 


o o 


T- 00 


2 

14 


4 

25 


00 00 


t o 




















Clarifying 

Questions 


NA 


o o 


o o 


o o 


o o 


o o 


o o 


o o 


o o 


Simple 

Statements 


NA 


2 

25 


1 

33.3 


18 

44 


3 

50 


7 

50 


4 

29 


4 

25 


CO CO 


Substantiated 

Statements 


NA 


1 

12.5 


o o 


4 

10 




o o 


o o 


o o 


CO CO 






















w 




















g 1 

is 
1 i 


A 1 


A 2 


B 3 


B 4 


C 5 


9 0 


D 7 


D 8 


Total 



CO 
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Notes 



* See Chicago Public Schools website: www cps.kl2.il.us 

^ Lee, Harper. I960. To Kill A Mockingbird. New York: Warner Books, Inc. 

^ For studies that have found that teachers alter their curriculum to meet expectation set by standards and 
assessments see Darling-Hammond, L. and Wise, A. 1985. Beyond Standardization; State Standards and 
School Improvement. The Elementary SchoolJoumal, 85, 3, 315-336.; Wilson and Corbett: 1990; 
Johnson, S. M. 1990. Teachers at Work. New York; Basic Books; Archbald, D.A. and Porter, A.C. 1994. 
Curriculum control and teachersCperceptions of autonomy and satisfaction. Educational Evaluation and 
Policy Analysis, 16, 21-39.; Wong, K. K., Anagnostopoulos, D., Rutledge, S,, Lynn, L, and Dreeben , R. 
1999. Implementation of An Educational Accountability Agenda: Integrated Governance in the Chicago 
Public Schools Biters its Fourth Year. A Report from the Department of Education and Irving B. Harris 
Graduate School of Public Policy Studies, University of Chicago. Studies have also found that standards 
and assessments result in a curriculum that is less diverse (Smith, M L. 1991 . Putting to the Test: The 
Effects of External Testing on Teachers. Educational Researcher. June-July, 8-11). These studies, 
however, have not looked at students Dexposure to higher level thinking skills in high stakes contexts. 

^ At School A, we interviewed eleven English teachers (65% of the department), and 9 mathematics 
teachers (64% of the department). At School B, we interviewed 10 English teachers (67%) and 12 
mathematics teachers (100%). At School C, we interviewed 7 English teachers (37%) and 9 math teachers 
(75%). At School D, we interviewed 7 English teachers (63%) and math teachers (6%). 

^ In total, we collected approximately 62 hours of observations. We observed two teachers at each of the 
four schools over the course of their To Kill a Mockingbird unit. We observed Teachers 1 and 2 at School 
A for 539 and 367 minutes respectively. We observed Teachers 3 and 4 at School B for 434 and 383 
minutes respectively. We observed Teachers 5 and 6 at School C for 376 and 346 minutes respectively. 
And at School D, we observed Teachers 7 and 8 for 752 and 485 minutes respectively. 
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Learned from Two Low-performing Chicago High Schools. Educational Policy 12: 1 & 2, pp. 31-47. 
Wong K., Anagnostopoulos, D., Rutledge, S., Lynn, L., and Dreeben, R, Implementation of an 

Educational Accountability Agenda: Integrated Governance in the Chicago Public Schools Enters its 
Fourth Year. Department of Education and Harris Graduate School of Pid)lic Policy Studies, University of 
Chicago: Chicago, IL. 

^ Wong K., Anagnostopoulos, D., Rutledge, S., Lynn, L., and Dreeben, R (1999) Implementation of an 
Educational Accountability Agenda: Integrated Governance in the Chicago Public Schools Enters its 
Fourth Year. Department of Education and Harris Graduate School of Public Policy Studies, University of 
Chicago: Chicago, IL. 

* Wong K., Anagnostopoulos, D., Rutledge, S., Lynn, L., and Dreeben, R (1999) Implementation of an 
Educational Accountability Agenda: Integrated Governance in the Chicago Public Schools Enters its 
Fourth Year. Department of Education and Harris Graduate School of Public Policy Studies, University of 
Chicago; Chicago, IL. 

^ Wong, K.K. 1990. City Choices: Education and Housing. Albany: State University of New York; 
Peterson, P., Rabe, B., and Wong, K. 1986. When Federalism Works. Washington, D C.: Brookings 
Institute. Pressman, J.L. and Widlavsky, A Implementation. Berkeley: University of California Press. 
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